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Who Am I? 


Software Engineer at Red Hat, Inc. 

Developer of PulseAudio, Avahi and a few other Free Software 
projects 

http://Opointer.de/lennart/ 
lennartQpoettering.net 
IRC: mezcalero 


< 3 ► <-► < 5 ► = 'Qq,O 


Lennart Poettering 


Pro Audio is Easy, Consumer Audio is Hard 




Not musician 
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Not musician 

Didn’t graduate in signal processing 



Why? 
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Correct misconceptions 
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Why? 

Correct misconceptions 

Improve cooperation, share code and interfaces 
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Correct misconceptions 

Improve cooperation, share code and interfaces 
It might just be interesting 



General Background 
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General Background 
Nobody makes money off Linux Audio 



General Background 

Nobody makes money off Linux Audio 

Only 3 people payed to hack on general purpose infrastructure by 
the big distributors 



However, we have allies, particularly in RT, Mobile, also V4L 



Pro Audio: knowledgable user 



Pro Audio: knowledgable user 
Consumer Audio: laymen users 
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Pro Audio: good hardware 
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Pro Audio: good hardware 

Consumer Audio: crappy hardware (Both the PC, and the Audio 
HW, Feature Monsters) 
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Pro Audio: Audio is primary purpose of the machine 
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Pro Audio: Audio is primary purpose of the machine 
Consumer Audio: Audio is one thing among many 



Pro Audio: User owns the machine, full configurability, user == 

administrator 
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Pro Audio: User owns the machine, full configurability, user == 

administrator 

Consumer Audio: user != administrator 
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Different Optimization Targets 



Different Optimization Targets: 

Pro Audio: low latencies, functionality 



Different Optimization Targets: 

Pro Audio: low latencies, functionality 

Consumer Audio: low power usage, dynamically changing 
environments, robustness 
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Latency vs. Power Usage 
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In Pro Audio: minimize latency at price of power usage, and 
dynamic configuration changes 
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In Pro Audio: minimize latency at price of power usage, and 
dynamic configuration changes 

In Consumer Audio: choose latency as high as possible, and as low 
as necessary, allow configuration changes 
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Result for consumer: we need to change latencies 



Result for consumer: we need to change latencies 
Unfortunately, most hardware/drivers cannot do that 
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Result for consumer: we need to change latencies 

Unfortunately, most hardware/drivers cannot do that 

Also: hardware/drivers limited in there range: ideally we have one 

wakeup every Is 
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Hence: schedule audio with system timers 



Hence: schedule audio with system timers 

That's difficult, because system timers deviate from sound card 
timers, and sound card timers are crap 
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Time interpolation 
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Time interpolation 

Linear Regression + Spline interpolation 
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Time interpolation 

Linear Regression + Spline interpolation 
Exponentional backoff 
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Time interpolation 

Linear Regression + Spline interpolation 
Exponentional backoff 
Everything twice 
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To make long latencies useful: support seeking 



To make long latencies useful: support seeking 
Requires history, for remixing 
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Dropouts, require dynamic adjustments of sleep times 



Timing and transfer block granularity 



Outlook, moving timing into the kernel? 



Non-interpolated timing has advantages? 



Non-interpolated timing has advantages? 
Sound card IRQs are accurate! 



Algorithm to use? DLL? 
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We cannot trust clients, one misbehaving client may not be 
allowed to stall the entire pipeline 
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RT CPU time calculations change, need to preemptively increase 
sleep times when streams are moved, removed and added 



RT CPU time calculations change, need to preemptively increase 
sleep times when streams are moved, removed and added 

Why? Never drop out 



Unreliable latencies, dynamically changing 



Compromises in RT code, 10 loops are not constant time 



Multiple encoded formats, conversion/resampling required, but 
ideally zero-copy 



Multiple encoded formats, conversion/resampling required, but 
ideally zero-copy 

Need to support S16LE and FLOAT32, and a lot more 



Large latency means exchanging of large blobs 



Large latency means exchanging of large blobs 
Zero copy (because sizeof(metadata) j sizeof(payload)) 



Cached audio samples 
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Cached audio samples 
Means ref counting 
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Cannot expose low-level hw controls 



Cannot expose low-level hw controls 
Need to merge and extend in software 



Cannot expose low-level hw controls 
Need to merge and extend in software 
Difficult: Routing in the Sound HW seldom known (USB!) 



Common: unreliable dB data 



Common: unreliable dB data 
If the mixer controls even exist 
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Machine not owned by the user and user not knowledgable 



Machine not owned by the user and user not knowledgable 
Means no low-level control on RT =i rtkit 



No memory locking 
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Compromises, compromises, compromises 



Cooperation PA, JACK 
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Use more standard desktop infrastructure. Tear down the 
seperating walls. 



Cooperation on APIs? 
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Cooperation on Timing 



Cooperation on Protocols? RTP? 



Come to LPC! 
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Come to LPC! 

Meet RT folks, V4L folks, MM folks 
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That's all, folks. 
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That's all, folks. 
Any questions? 
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